Data Parallel Design Patterns

Data Parallelism has moved to the forefront of parallel computing. This is driven in part by hardware trends. Vector instructions have been integrated into the instruction sets in most commercially important CPUs. GPUs memory hierarchies and floating point capabilities make them suitable for applications well beyond graphics. At a more fundamental level, however, the increasing emphasis on data parallelism emerges from the need to find more fine grained tasks in a problem. As core counts climb, we can no longer achieve the scalability we need through coarse grained tasks.